Thesis Proposal Learning optimal policies for compliant gaits and their implementation on robot hardware

Bipedal animals exhibit a diverse range of gaits and gait transitions, which can robustly travel over terrains of varying grade, roughness, and compliance. Bipedal robots should be capable of the same. Despite these clear goals, stateof-the-art humanoid robots have not yet demonstrated locomotion behaviors that are as robust or varied as those of humans and animals. Current modelbased controllers for bipedal locomotion target individual gaits rather than realizing walking and running behaviors within a single framework. Recently, researchers have proposed using the spring mass model (SMM) as a compliant locomotion paradigm to create a unified controller for multiple gaits. Initial studies have revealed that policies exist for the SMM that exhibit diverse behaviors including walking, running, and transitions between them. However, many of these control laws are designed empirically and do not necessarily maximize robustness. Furthermore, the vast majority of these controllers have not yet been demonstrated on physical hardware, so their utility for real-world machines remains unclear. This thesis will investigate gait transition policies for the SMM that maximize an objective measure of robustness. We hypothesize that these control policies exist within the SMM framework and can be numerically calculated with guaranteed optimality and convergence. Specifically, we aim to investigate the following two claims. 1) All proposed SMM gait transition policies can be computed using reinforcement learning techniques with linear function approximators. 2) This method can generate new policies which maximize the basin of attraction between walking and running states. Initial results show that these reinforcement learning methods can indeed learn existing SMM policies previously found through Poincaré analysis. If these algorithms are successful in finding globally optimal policies, they may lead to bipedal locomotion controllers with both diverse behaviors and largely improved robustness. We will experimentally evaluate the utility of these control policies for human-scale bipedal robots. This thesis will extend our analysis of SMM policies on the ATRIAS robot platform to include multiple gaits and gait transitions. Our initial hardware implementation of SMM running has revealed two technical challenges we will address. 1) Modeling errors for both the simplified model and higher-order robot lead to performance degradation from simulation. We will investigate improving this with online methods of parameter estimation and learning. 2) Our experiments have only evaluated planar running and must be extended to include 3D locomotion. If these two challenges are overcome we will have experimentally evaluated SMM running, walking, and transitions between on a physical bipedal robot.